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ABSTRACT 



As part of its contract to develop a framework for 
continuous school improvement in its four- state region (Kentucky, Tennessee, 
Virginia, and West Virginia) , Appalachia Educational Laboratory staff 
designed the Quest project. Based upon principles of inquiry, collaboration, 
and action research. Quest supports and investigates ongoing school 
improvement efforts through rallies, summer symposia, a Scholars program, 
visits to participating schools, communication via listserv and mailings, and 
the creation of a Quest network of schools. This report describes and 
assesses an elementary school network rally and a Quest Scholars meeting held 
in February 1999 in Lexington, Kentucky. Teams of students, teachers, 
parents, and administrators from each of six elementary schools attended the 
-rally, for a total of 28 participants, and 5 Quest Scholars attended their 
meeting. Rally activities included a demonstration of the California 
Protocol, an interactive process to engage the entire school in the school 
improvement process; a session on brain-based learning; and alignment of 
teams' school improvement plans with Quest support activities. Evaluation 
data were generated by evaluator participant observation, unstructured 
interviews, and three feedback forms. Data indicate that the rally's six 
goals- -connect with colleagues, create a learning community, connect with 
concepts related to continuous school improvement, create personal and shared 
meaning, commit to continue learning with this community, and commit to 
continue this Quest back home- -were met very well; connecting with colleagues 
and commitment to the Quest learning community were achieved best. Appendices 
include the Quest brochure and framework for continuous improvement, feedback 
forms, information on the Protocol process, and the evaluation standards 
checklist. Contains 19 references. (Author/TD) 
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EXECUTIVE SUMMARY 



As part of its contract to develop a fiamework for continuous school improvement in its four- 
state region, Appalachia Educational Laboratory (AEL) staff designed the Quest project. Based 
upon principles of inquiry, collaboration, and action research. Quest proposes to support and 
investigate ongoing school improvement efforts through twice-yearly conferences (which staff 
renamed rallies), summer symposia, a Scholars program, visits to participating schools, 
communication via listserv and mailings, and the creation of a Quest network of schools. This 
evaluation report describes and assesses one elementary school network rally, convened February 
22-23, 1999, in Lexington, Kentucky. In addition, this report documents a three-hour Quest Scholars 
meeting held on February 21, 1999. 

Twenty-eight participants attended, as well as four AEL staff members. Six elementary 
network schools sent teams ranging from two to seven members. One school also brought four 
students to participate in a demonstration of the Protocol, a technique for examining student work, 
although they did not take part in other rally activities. Eighteen participants were teachers, five 
were parents, and five were school building administrators. Seven of the 21 respondents to a final 
evaluation form indicated having attended three or more Quest events, ten had attended one or two 
events, and four had attended no other project gatherings. Of the 28 participants, five were men and 
23 were women. Three might be considered African American, one Latina, and the remainder white. 

The rally was evaluated in terms of whether, and to what extent, the conference goals were 
met. To this end, a variety of data were gathered: field notes were taken during evaluator participant 
observation of all conference activities, participants completed three feedback forms, and 
unstructured interviews were conducted throughout the rally. The rally goals included: (1) to 
connect with colleagues, (2) to create a learning community, (3) to connect with concepts related to 
continuous school improvement, (4) to create personal and shared meaning, (5) to commit to 
continue learning with the Quest community, and (6) to commit to continue the Quest back home. 

Analysis of the feedback data revealed that participants thought the rally goals had been very 
well met. On a 5-point Likert-type scale, mean ratings of the degree to which goals were achieved 
ranged from 4.65 to 4.40. Most well-achieved, with identical mean ratings of 4.65, were the goals 
concerning connecting with colleagues and committing to continue learning with the Quest 
community. Less well-achieved was the goal concerning commitment to continue the Quest back 
home, although with a mean rating of 4.40, respondents nonetheless indicated that the goal had been 
met. Qualitative data tended to support the positive quantitative assessments of the rally. 

Based upon the data, the evaluator concluded that the rally had continued to provide support, 
encouragement, and collegial connection to those undertaking continuous school improvement. 
Recommendations included offering techniques and support for specific improvement efforts at 
network schools, further structured and informal means for network communication, assessing 
participant information needs, and ensuring adequate time for school team planning. 

ii 
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INTRODUCTION 



As part of its contract to develop a framework for continuous school improvement in its four- 
state region, Appalachia Educational Laboratory (AEL) staff designed the Quest project (see 
Appendix A). Based upon principles of inquiry, collaboration, and action research. Quest proposes 
to support and investigate ongoing school improvement efforts through twice-yearly conferences 
(which staff renamed rallies), summer symposia, a Scholars program, visits to participating schools, 
communication via listserv and mailings, and the creation of a Quest network of schools. 

In the summer of 1996, Quest staff at AEL began working with teams from school 
communities in three West Virginia county school districts to invigorate efforts for continuous 
school improvement, using a variety of techniques for gathering input from all those with a stake in 
their local schools (Howley-Rowe, 1998g). This first “learning community,” called Leadership to 
Unify School Improvement Efforts (LUSIE), was comprised of school teams including students, 
teachers, administrators, parents, and community members. Ultimately, this group wrote individual 
school visions and improvement plans, and co-authored (with AEL) Creating Energy for School 
Improvement (1997), a supplemental guide for those poised to write their own state-mandated school 
improvement plans. 

Quest staff also were committed to creating learning communities devoted to exploring 
continuous school improvement across the AEL region of Kentucky, Tennessee, Virginia, and West 
Virginia. Hence, staff scheduled a pilot Inquiry Into Improvement conference in April 1997 for 
selected region high schools. Schools were selected in several ways. Some schools were 
recommended for the Quest experience by central office staff or school administrators. Other 
schools were asked to join Quest because they had participated in previous AEL programs. Still 
other schools were invited because Quest staff believed they were primed for the kind of 
collaborative inquiries into school improvement that Quest was designed to provide. 

In October 1997, in Roanoke, Virginia, another conference was held for designated high 
schools in AEL’s region, this time with an explicit emphasis on forming and nurturing a network of 
schools (Howley-Rowe, 1998c). A similar conference was held in Nashville, Tennessee, for 
designated region elementary schools in November 1997 (Howley-Rowe, 1998a). In order to 
facilitate the development of a Quest school network and to continue to help invigorate continuous 
school improvement efforts within network schools, staff planned a sequence of events in 1998 
following these initial conferences. Dissatisfied with the conventional and prescriptive cormotation 
of “conference,” Quest staff chose to call these network meetings “rallies.” Thus, all events 
previously called conferences are now termed "rallies.” 

The high school network met a second time on February 8-10, 1998, at the Pipestem State 
Park Resort in West Virginia (Howley-Rowe, 1998d), following which the elementary school 
network participated in a rally on February 22-24, 1998, in Lexington, Kentucky (Howley-Rowe, 
1998b). During the summer, 11 network members participated in the Quest Scholars Program, 
meeting at a colloquium in Charleston, West Virginia, on July 16-18, 1998, to collaborate with 
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project staff in ongoing efforts to conceptualize, design, and research Quest (Howley-Rowe, 1998e). 
And in August, network members and other educators in AEL’s region participated in a symposium 
on assessment of student work (Howley-Rowe, 1998f). 

From the high school network rally in October 1997 to the August 1998 summer symposium, 
the Quest network contained an essentially stable membership, although there were differences in 
the number of school teams attending each event and in the frequency that school teams attended 
gatherings, a phenomenon project staff have undertaken to study. During this period. Quest staff 
hosted six network events. 

Beginning their second year of network activity. Quest staff invited the elementary and high 
school networks to attend a rally together on November 2-3, 1998, at the Glade Springs Resort, near 
Daniels, West Virginia (Howley-Rowe, 1999a). Approximately half of the Quest Scholars met on 
November 1, 1998, to plan with project staff several rally activities. Scholars from the high school 
network met for three hours on February 14, 1999, prior to a high school network rally held on 
February 15-16 in Roanoke, Virginia (Howley-Rowe, 1999b). A similar rally was held for 
elementary network members on February 22-23, 1999, in Lexington, Kentucky. This report 
describes the elementary level Scholars meeting and summarizes evaluation of the elementary school 
rally. 



The primary audience for this report is Quest staff. It is intended to provide information to 
staff as they make decisions about future rallies and the development of the network. In addition, 
this report will be part of an ongoing series of reports about Quest events (Howley-Rowe, 1998a-g, 
1999a-b). This series will document the evolution of the Quest network and the process whereby 
staff strive to enable continuous school improvement. Consequently, this report may also prove 
useful to others interested in building networks or promoting school improvement over time. 

One purpose of this report is to assess whether, and to what extent, rally goals were met. The 
six major goals discussed in earlier reports are also analyzed here: (1) to connect with colleagues, 
(2) to create a learning community, (3) to connect with concepts related to continuous school 
improvement, (4) to create personal and shared meaning, (5) to commit to continue learning with 
the Quest community, and (6) to commit to continue the Quest back home. In addition, description 
and analysis of the rally provided in this report contribute to ongoing documentation of the Quest 
project and of the development of the Quest network. 




9 



3 



METHODOLOGY 



The methods used for this evaluation component of the Quest project were primarily 
qualitative, although some quantitative data were also collected and analyzed. During the rally, the 
evaluator engaged in participant observation (Becker & Geer, 1957; Emerson, 1983; Glazer, 1972; 
Miles & Huberman, 1994; Patton, 1980), a method highly suited “for studying processes, 
relationships among people and events, the organization of people and events, continuities over time, 
and patterns” (Jorgensen, 1989, p. 12). Furthermore, consistent with the Quest paradigm, participant 
observation involves “a flexible, open-ended, opportunistic process and logic of inquiry through 
which what is studied constantly is subject to redefinition based on field experience and observation” 
(Jorgensen, 1989, p. 23). This method “is a commitment to adopt the perspective of those studied 
by sharing in their . . . experiences” (Denzin, 1989, p. 156), thereby enabling researchers to evaluate 
how an event or process appears and feels to participants. By “exploit[ing] the capacity that any 
social actor possesses for learning new cultures, and the objectivity to which this process gives rise,” 
participant observation further produces data that is both rich and valid (Hammersley & Atkinson, 
1983, p. 8). 

During this rally, the evaluator played a role more akin to what Denzin typifies as “observer 
as participant,” rather than as a complete participant observer (1989). That is, the evaluator’s contact 
with rally attendees was not as a participant in the activities in which they were engaged, but instead 
as a roaming onlooker and occasional conversationalist. The evaluator sat in on participant group 
endeavors, watched the large group as the rally unfolded, shared evening entertainment activities, 
and took advantage of serendipitous occasions to chat with participants. 

In order to corroborate the theses generated by participant observation, the evaluator also 
analyzed data from the feedback forms designed by Quest staff soliciting participant assessment of 
the process (see Appendix B). Using several data sources in order to corroborate theses is what 
Brewer and Hunter (1989) call “multimethod research” or “triangulation.” This approach posits that 
the strengths of each method will compensate for the weaknesses in others, ultimately providing a 
more complete account of that being studied. Hence, in addition to participant observation, three 
evaluation forms were used to collect further information. One feedback form asked attendees to 
discuss their experiences during the first day of the rally. This form asked participants to record: 
“Learnings, insights, ah-ha’s from the day,” “Ways in which I contributed,” “Things I want to 
explore further,” “Things that worked especially well for me,” “Things that would have allowed me 
to contribute more,” and “Things to trash.” Another feedback form was distributed at the end of the 
rally and asked participants for their evaluations of specific activities, including demonstration of 
the California Protocol, review of network schools’ improvement plans, review of brain-based 
learning research, and use of the Quest self-scoring profile. This form also included a quantitative 
assessment of the degree to which participants thought the rally goals had been achieved, using a 5- 
point Likert scale. A third form posed open-ended questions concerning specific examples of ways 
each of the rally goals had been met. 
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A third data collection method included unstructured interviews conducted during the course 
of the rally. As opportunities arose for relatively private conversation, participants were asked to 
discuss their assessments of the rally generally. Interview responses were later coded and analyzed 
by theme. 

Analyses of participant observation field notes, interview data, and evaluation form 
responses were made by theme or question, as most appropriate. Themes were coded, and responses 
within each theme were tabulated. Field notes were condensed for inclusion in the descriptive 
section of this report. 

Pseudonyms are used throughout this report for participant and school names in order to 
protect the anonymity and confidentiality of those involved in the Quest network. 
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SCHOLARS MEETING 



Five elementary school network Quest Scholars attended the February 21 meeting facilitated 
by two project staff members. Four Scholars were women, one a man. Two were principals, and 
three teachers. One might be considered African American, the remainder white. 

The Quest Scholars were to meet at the Wyndham Garden Hotel in Lexington, Kentucky 
from 2:30 until 5:50 p.m. in a small meeting room. However, project staff did not begin the 
proceedings until 3:07 because two of the five Scholars were late arriving. In the interim, staff and 
participants chatted, joked, and snacked informally as a Mozart CD played in the background. 

One of the latecomers arrived at 2:53, and staff decided to begin the meeting officially at 
3:07. A facilitator wrote the topics to be addressed on the marker board at the front of the room. 
Topics were Quest creed— where do we go from here; school journal (portfolio) 
development— documentation of journey, data collection; engagement study; summer symposium; 
Scholars colloquium; Scholars program 1999-2000; and agenda for the rally. As she did so, the 
Scholars continued to chat informally, one asking a project staff member about “our high school 
friends." 

Just as a facilitator began to describe the agenda, the second latecomer arrived, laughingly 
sharing her tale of getting lost along her journey to the hotel. Once she settled into a chair, the 
facilitator began again, discussing the history of the creeds developed by participants of the two 
networks. The facilitator added that her purpose in discussing the creeds was, "We are bringing them 
back to the Scholars to ask what’s next.” She went on to describe the high school network Scholars’ 
preference for a combined creed and their suggestion of a technique for assessing this possibility 
across both networks. 

Asked for their feedback about this, the Scholars indicated their agreement. One Scholar 
said, “It seems we should be able to agree on common [ground].” Another Scholar remembered a 
suggestion from participants at the combined rally in November 1998 to construct a combined creed 
with subsections specific to each network. The Scholars then discussed their concerns that the 
creeds were too long and lacked “a language common to all, including parents and kids.” One 
participant summed up their conversation, offering that the creeds “need[ed] to be combined and 
shortened.” 

Beginning at 3:25, Quest staff recounted decisions made thus far about schools’ 
documentation of their growth during participation in the network in “journals] of the journey,” 
which they hoped to publish in a book about the project. Such documentation would also “enable 
evaluation of Quest, especially student outcomes.” They added that there was “no prescribed 
form[s]” for such documentation, but that they “have come up with a philosophy and format . . . that 
your involvement in Quest should enhance what you were doing anyway at your school.” They then 
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went on to share a planning worksheet they had prepared to assist teams align their local school 
improvement goals with Quest activities and to collect data about their progress toward such goals. 
“[We] see this as a planning sheet that would eventuate into this journal, or log of your journey,” 
documenting school improvement for each school and “help[ing] to meet the project’s need for 
evaluation.” Quest staff also noted that this effort was encouraged by Scholars’ recommendation that 
schools be held more accountable to the project for their improvement work. 

Scholars were asked for their reaction to the worksheet. One replied, “One of the things 
Quest does for us is give us a retreat atmosphere ... an opportunity to focus energy on the task at 
hand without phones ringing or needing to get home because the babysitter’s time is up ... [I think] 
this will help us embellish the integrity of our three goals in our plan.” Quest staff replied that they 
hoped Quest “is value-added for you.” 

The Scholars then discussed in more detail their appreciation of the project. Offered one, 
“The readings help us see things in a different way . . . Wheatley has been really significant to us.” 
She added, “One of the most difficult things to do in a social organization is to show cause and effect 
relationships,” referring to plans for schools to document their Quest journeys. A facilitator replied, 
“Well, Wheatley says that there are no cause and effect relationships. We don’t believe everything 
in your school is directly because of Quest.” Another Scholar offered that a “good thing about Quest 
is it regenerates .... [It] confirms you because you [Quest staff] come here, you’ve read so much 
and know what you’re doing, and look at us and say we’re on the right track ... and it is confirming.” 

A facilitator reoriented the discussion, first praising the diversity of Scholar perspectives and 
then adding her hope that “school teams will really get into it [documenting their growth] .... [We] 
hope that you will see what you can notch up in your plans.” 

One Scholar voiced his concern: “My problem with this is being able to tie two separate, 
segments. On one side is school culture, community. Then on the other side you have what people 

in the community are really looking at— test scores I’m not so sure if we did every one of these 

[activities designed to address Quest framework components] . . . test scores might not reflect this.” 
He contributed an anecdote about test scores dropping after implementation of an inquiry-based 
science program. A facilitator asked how long the program was implemented; the Scholar replied 
that they continued to use the program, noting, “Scores are up but [it’s] because we have pulled back 
on some parts of inquiry.” 

The Scholars then discussed their perspectives on the connection between achievement and 
instruction, classroom climate, and student motivation, and the disjunction between test-taking and 
meaningful learning. A facilitator acknowledged the Scholars’ concerns, saying, “It will be a 
challenge [to link student achievement to activities related to Quest]. But we hope we can think 
together about how these things make a difference.” 




13 



7 



Staff next described their small study of what influences schools’ engagement with the Quest 
network, aimouncing that participants would be asked to take part in focus group interviews the next 
day. The summer symposium, to be held July on 26-27 in Gatlinburg, Tennessee, was also 
announced. Staff described the various topics they were considering for symposium sessions: 
student motivation, project-based learning. Interdisciplinary Teamed Instruction (ITI), active 
learning strategies, emotional intelligence, student-led conferences, brain-based learning principles, 
and a possible poster session for network schools to present their efforts. Scholars discussed the 
issues of depth versus breadth of topics, and the facilitators voiced their anticipation that Scholars 
would take an active part in planning and presenting sessions at the symposium. 

The Scholars colloquium was also announced. To be held from July 12-14 at Mountain 
Lake, Virginia, the colloquium this year would differ significantly from the first colloquium, in that 
“[We] hope there will be time to do some thinking and writing for a book modeled somewhat on 
Senge’s Fifth Discipline Fieldbook . . . with some theory, the framework, stories.” “This is all to 
say,” added one Quest staff member, “that we want a retreat setting ... and do you want to re-up 
[your participation]?” Two Scholars mentioned schedule conflicts they might confront, and Quest 
staff noted these. 

A facilitator then asked Scholars their opinion of the book they were planning. One replied 
that it sounded “exciting.” Another said, “The synergism of putting the group in one place, 
distraction free, would result in a product better than what I might try to pull together myself” 

Then, a staff member introduced a new Quest tradition, the Giraffe Award, and described the 
book from which she had drawn the inspiration for it. She noted that giraffes were symbolic in 
several ways: they take risks (“sticking out one’s neck”); they are oriented toward community (being 
animals who travel in herds); they stop to graze, symbolizing reflection along one’s journey; they 
are able to survey the “big picture” from their lofty vantage point; and they stand out in a crowd, 
again due to their height. After describing procedures for nominating participants for the award, she 
then quickly described the rally agenda and discussed the evening’s dinner plans. 

The group then relaxed for a few moments together, sharing stories and generally catching 
up with each other. One Scholar asked another for information about multi-age classrooms at her 
school and began to arrange a site visit for observation. The group dispersed at 5:30. 
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RALLY ACTIVITIES 
Day One 



The elementaiy school network rally began at 8:00 a.m. February 22, 1999, at the Wyndham 
Garden Hotel in Lexington, Kentucky. Twenty-eight participants attended, as well as four AEL staff 
members. Six elementary network schools sent teams ranging from two to seven members. One 
school also brought four students to participate in a demonstration of the Protocol, a technique for 
examining student work, although they did not take part in other rally activities. Of the participants, 
18 were teachers, five were parents, and five were school building administrators. Responses to a 
query on one of the final evaluation forms concerning the number of Quest events previously 
attended indicated that seven of the 21 respondents had attended three or more; ten had attended one 
or two events; and four had attended no other project gatherings. Of the 28 participants, five were 
men and 23 were women. Three might be considered African American, one Latina, and the 
remainder white. 

The rally activities took place in a conference room in the hotel. At the front of the room 
were tables arranged perpendicularly to each other and at an angle from one comer. On the tables 
were paper, pens, tape, scissors, reading materials, and an overhead projector. Participants sat at 
round tables clustered near the front of the room. On the tables were baskets filled with supplies and 
bags with various toys, as well as pots of African violets. Throughout the rally, the room 
temperature fluctuated between uncomfortably cold and hot. Facilitators made repeated efforts to 
convince hotel staff to adjust the thermostat adequately, but the temperature did not stabilize well 
into the second day of activities. 

After welcoming attendees at 8:03 a.m., the rally facilitators invited participants to engage 
in an introductory activity. In the activity, attendees were to find partners with whom to share 
accounts of occurrences at their schools that had generated excitement or energy. Participants stood 
up from the tables at which they had been seated, made eye contact with potential partners, and then 
met to discuss their accounts. The conference room filled with conversation from 8:15 imtil 8:25, 
when the facilitators requested that participants return to their seats and write the stories they shared 
on large index cards. 

After 1 0 minutes of quiet writing, attendees were asked to decide with which of the six Quest 
framework components their stories most closely corresponded. A facilitator briefly reviewed the 
framework. Then, participants were requested to write their names on small Post-it note paper and 
place the paper on a depiction of the framework component most closely associated with their 
stories. Attendees placed most Post-it notes on the framework components of enabling SMART 
learners and broadening the learning community, while only one Post-it note was placed on the 
component of strengthening the learning culture. 
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At 8:50, attendees were asked to share their stories with their school team members, and then 
designate one member to sign the large Quest time line and write a brief description of recent school 
accomplishments. The facilitators collected the cards on which participants had written their stories 
at 8:56 and next requested that a member from each team report on activities at their school since 
the last rally. 

The first school to report noted that student-led parent-teacher conferences had been 
expanded to include third through fifth grades, with much success. The award of an approximately 
$300,000 21st Century Community Learning Center grant was the major event at another school. 
Team members reported that their small, rural school was now able to offer an after-school program 
four days per week, including piano, gymnastics, karate, and basketball instruction, through the grant 
award. Community support for the endeavor, team members added, was evident in that parents 
provided transportation for their children from the after-school program despite the obstacles in so 
doing that many poor, rural residents confront. A third school announced that they had received 
ftmding for a new addition to their facility. Other events included a Quest co-venture during which 
parents participated in Interview Design, the institution of portfolio use, and a “great science fair.” 

A fourth school reported that they continued to use the Quest framework component of 
enabling SMART learners as a school theme. In addition, they were chosen as a Blue Ribbon 
school, received double KIRIS rewards, and had sent their jump rope team on tour to North Carolina. 
Following this report, one of the facilitators requested that the team present a video they, students, 
and a parent employed by the local public broadcasting station produced about their school. The 
video, of high quality, depicted students and faculty discussing the quality of education received at 
the school. Participants applauded the video enthusiastically afterwards. 

Three grants were awarded another school for the expansion of their computer and 
technology lab. School community members were planning to create a digital presentation for open- 
house activities using their new equipment, and in order to integrate the new technology into their 
teaching, teachers were required to write at least four lesson plans with technology explicitly 
included. A final school team reported their preparations for their annual community festival, the 
success of their second year in a K-12 consolidated school building, their use of money saved 
through consolidation for community enhancement activities such as supplementation of field trips 
and balloons for hospitalized students, and the acquisition of many new computers. 

At 9:30, a facilitator introduced a “new Quest tradition," the Giraffe Award, amidst some 
participant laughter. Giraffes, she explained, were symbolic in several ways: they take risks 
(“sticking out one’s neck”); they are oriented toward community (being animals who travel in herds); 
they stop to graze, symbolizing reflection along one’s journey; they are able to survey the “big 
picture” from their lofty vantage point; and they stand out in a crowd, again due to their height. 
Participants were then asked to nominate another network member for the award who evinced the 
giraffe-like quahties noted. To do this, attendees wrote nominees’ names on index cards, which were 
placed in a small box. A card was to be drawn from the box at the end of the day’s activities. 
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The facilitators previewed the rally agenda and activities beginning at 9:34. They noted that 
schools would share their stories and experiences; that Quest staff would review the Quest 
framework and a self-assessment instrument based on the framework, and offer an orientation 
session for participants new to the network; that attendees would learn together about brain-based 
learning research and the Protocol process; and that school teams would be provided time for action 
planning. Following this explanation, attendees were given a break from 9:40 until 9:50. 

At 9:50, a Quest staff member talked about her involvement with the School Change 
Collaborative (SCC), a collaborative group coordinated by the Northwest Regional Educational 
Laboratory (NWREL) interested in part by the challenge of bringing student voice into the change 
process and promoting the growth of learning communities within schools. She added that through 
the SCC she had learned of the Protocol process, and then she introduced Fred', an elementary 
school principal who decided to use the process in his school. 

Fred began his discussion with an allegory about turkeys receiving a day of staff 
development to learn how to fly. By the end of the day, he continued, the turkeys had gained 
confidence and were flying all around the room; afterwards, they walked home. Participants laughed 
loudly at Fred’s story. Then Fred went on to say that what had sfruck him about the Protocol process 
was that it offered a means for examining and discussing student work. In all his time as an 
educator, from teacher to central office staff, he said he “had never really examined actual pieces of 
student work!” Although he did not immediately find the Protocol process compelling and applicable 
to his school, he later decided to pilot the process with his staff to discuss writing instruction and 
assessment. He added that the benefits of the process included “forced reflection and [it] forces 
[you] to stay on task.” 

Using an overhead projector, Fred then explained to participants the Protocol process, roles, 
and assumptions (see Appendix C) and introduced several teachers from his school who would next 
model the process. At 10:14, four teachers convened at the front of the conference room, two of 
whom would undertake the role of analysts, or discussants, while the other two would act as 
reflectors, providing both supportive and more critical feedback to the analysts. The two analysts 
shared samples of student writing with the other Quest participants, and then went on to discuss 
whether and how the samples had improved over time. The analysts spoke quietly, however, making 
it difficult for some attendees seated near the back of the room to hear them. At one point, one such 
attendee requested they speak more loudly; after a few moments of more audible speaking, the 
analysts returned to their quieter discussion, however. 



'Pseudonyms are used throughout this report in place of personal and school names in 
order to protect the anonymity and confidentiality of participants. 
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The demonstration of the Protocol process ended at 10:40, at which time a facilitator asked 
other attendees if they had any questions concerning the process. There were none, and so the 
facilitator requested that the four teachers who had demonstrated the process share some of their 
reflections on it. They noted that they had been reluctant to use the process at first, but that over 
time they saw that it enabled them to think more fully about writing and the structure of their writing 
assignments. Other benefits included “spend[ing] more time on writing” in the classroom and 
“sharing ideas with each other.” Fred asked the teachers what they thought the difference was 
between using the Protocol process and “just talking." They replied that it “forces you to talk and 
reflect” and it enables “better use of time.” 

Following this, other attendees asked several questions about the process and how it might 
be used in various situations. At one point, Fred noted that teachers at his school had asked if they 
were doing the Protocol correctly. In reply, he asked them whether they were thinking more about 
student writing than they ever had before. When they responded affirmatively, he told them that was 
what mattered. A facilitator brought the discussion to a close at 1 1 :01, with applause and a “thank 
you.” Attendees took a short break until 1 1 :09. 

Participants reconvened, moving their chairs into a large circle near the rear of the room as 
requested by Quest staff in order to observe four fourth grade students from Fred’s school participate 
in a version of the Protocol process. Fred introduced the demonstration, noting that use of the 
process with students began when a teacher, Rhonda, running the after-school program adapted it. 
In addition, the teacher had ensured that “diverse learners” participated. Fred then thanked the 
students for attending and turned over the proceedings to Rhonda. 

Rhonda introduced the students, two African American boys and girls, who were to 
demonstrate the Protocol process to discuss stories they had written. After two students, acting as 
analysts, read and briefly discussed their stories, Rhonda prompted them by asking them to answer 
the question, “How did I develop my main idea and content?” After replying to the question, the 
other two students, as reflectors, provided some warm feedback. Next, the students switched roles 
and repeated the process. Following this, Rhonda guided the students through an adaptation of the 
process, in which all four talked together about their stories and responded to her prompts for cool, 
or more critical, feedback. She also asked each student what grade they would give their own paper. 

The demonstration concluded at 1 1 :30, amidst applause and praise for the students. A 
facihtator then asked that the students participate in a Fishbowl process, sitting in a small discussion 
circle as others looked on. The facilitator explained that one chair in the circle was empty to allow 
one onlooker at a time to join the Fishbowl to ask a question of the participants. One of the 
facilitators began by asking “How has it felt for your classmates to read your work?” Replies 
included, “It’s fun. It helps me with ideas, so I feel better about what I wrote,” and “It lets me help 
others and others help me.” Another question was “When you’re doing a Protocol, have you ever 
thought that a story was not very good? Was it hard? What did you do?” Responses included, "I just 
told them,” “I gave them ideas out of what they had,” “Sometimes I kept it to myself,” and “Ms. 
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Effington helps us not be nervous.” A facilitator asked, “How does it feel when friends give you 
suggestions?” One student answered, ‘Tm glad. [I] always need some help, even if [I] don’t like [it] 
... to help my story be better.” Another replied, “[I] feel like other kids want our stories to be 
interesting.” The discussion concluded at 1 1 :50. 

A facilitator then distributed to the participants copies of a letter one of the students had 
written inviting them to attend an event at which Fred was to eat worms as an Accelerated Reader 
award. The student read the letter aloud, and a facilitator presented gifts to the students. 

At 1 1 :55, attendees were asked to return to their school teams to discuss ways in which they 
might use the Protocol process. At 12:00, lunch was served. 

The group reconvened at 1:07 p.m. Fred summed up his school’s experience with the 
Protocol process, reiterating that it had been useful to “diverse learners.” A facilitator commented 
that after observing the use of Protocol at Fred’s school, she realized that students there had begun 
to think of their writing as “their own.” Following this, school teams were allotted further time to 
discuss ways they might use the process in their own schools. 

At 1 :25, Quest staff asked whether school teams wished to share their ideas. Only one 
school team replied, saying they might use the process to discuss quality control of the products 
produced through their Microsociety program. 

From 1 :28 until 1 :40, attendees answered the Quest self-scoring profile. In addition, they 
circled items that were unclear and underlined ambiguous words or phrases. The room was quiet 
as participants worked. 

At 1:40, a Quest staff member explained that the profile might be useful to schools as a 
means to track their development in terms of each of the Quest framework components. She then 
directed attendees to convene in role-alike groups to get acquainted, discuss how results from 
administration of the self-scoring profile might be useful to their particular role, and talk about what 
items on the Profile were troubling or unclear. The various groups talked until 2:10, at which point 
they were asked to report their discussions to the larger group. 

Participants offered a number of comments about the self-scoring profile. Some teachers 
reported that they were uncomfortable using the instrument to evaluate their entire school. Others 
felt imequipped to provide responses to items asking them to assess whether “all” school community 
members engaged in various practices. One group of teachers suggested that the directions for 
completing the Profile include a note that the data thereby gathered were perceptual. A group of 
parents felt that parents completing the Profile had less information about their children’s school than 
teachers and administrators might, and thus would have difficulty answering some of the items. 
Another critique was that the questions were “too broad,” while yet another was that, if parents were 
asked to respond to the instrument, directions ought to clearly indicate for what purpose the data 
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were being collected. The use of educational jargon was likewise questioned by parents. 
Administrators expressed concerns about the length of the Profile, the language used, and the lack 
of any open-ended queries. But they also thought that the instrument would be useful to compare 
the perceptions of various subgroups within the school community as well as to make comparisons 
over time. 

The facilitators thanked participants for their comments, communicating their intention to 
revise the profile. At 2:35, attendees were requested to reconvene in their school teams to begin 
aligning their Quest endeavors with goals from their school improvement plans. First, however. 
Quest staff conducted an activity with balloons to re-energize participants; attendees appeared to 
enjoy the energizer, laughing enthusiastically and teasing one another. 

At 2:46, the facilitators provided the rationale for aligning school improvement plans with 
Quest activities: “We hope you never do something just because it is a Quest activity. We want the 
activities you conduct to be supported by Quest.” Then they asked that school teams examine their 
plans to determine “what in your plans can be supported by Quest.” The facilitators provided 30 
minutes for the activity and a worksheet to assist in the planning process. 

Teams worked intently, reading their improvement plans and discussing which goals might 
best correspond to Quest activities. Many teams began to complete the worksheet provided by Quest 
staff By 3:20, the facilitators asked participants to wrap up their discussions and added that time 
would be allotted the next day for continued planning. In addition, teams would be offered the 
opportunity to discuss their plans with another team using the Protocol process. 

. Participants were then asked to participate in one of three focus groups in order to provide 
data for Quest research concerning schools’ initial and sustained involvement in the network. 
Meanwhile, participants new to the Quest network participated in a more in-depth orientation to the 
project. Participants in the focus groups and the orientation session were then asked to complete an 
evaluation form concerning their experience of the first day of the rally. 

Following the focus groups and the introductory session, attendees reconvened in the 
conference room for the presentation of the Giraffe Award, amidst applause and apparent 
amusement. In addition, three team members from one school announced that they needed to return 
to their school to assist with the implementation of a new grant; the facilitators and other participants 
said good bye and then adjourned for the evening. 
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Day Two 



The second day of the rally began after breakfast at 8:01 with a welcome from the facilitators 
and some informal conversation about the previous night’s activities. Then at 8:10, the facilitators 
distributed a form with items from both the elementary and high school creed statements written at 
earlier rallies. The form requested that respondents rate the importance of each item. The items, 
however, were not printed in such a way as to reveal from which level creed statements they were 
drawn, as the purpose of the activity was to assess the relevance of each item regardless of level; 
participants had disagreed at the combined high school and elementary network rally in November, 
1998 about whether to create a combined creed for both networks. Quest staff noted that the 
Scholars intended to use the data gathered during this activity to revise the creeds. 

After attendees completed the activity, the facilitators briefly reviewed the agenda. At 8:21, 
participants were asked to coimt off by six in order to join one of six groups. Each group was to read 
an excerpt concerning one of the six Quest framework components and then plan a presentation 
about what the component might mean to schools. 

The six groups worked imtil 9:29, reading about their respective components, discussing the 
topic, and then planning their presentations. The presentations included a poem read in unison, a 
skit, a demonstration of a reading lesson, and a game. Attendees applauded following each 
presentation, and laughed during those that were intended to be amusing. 

, From 9:47 until 10:12, Quest staff discussed the Scholars program and the summer 
symposium. They communicated the dates for each, as well as the sessions likely to be offered at 
the symposium. Attendees were then given a break. 

Reconvening the group at 10:35, one of the facilitators asked school teams if they had 
formulated questions to address during the Protocol process to be conducted in the afternoon; few 
had. Next, after introducing some facts about the brain and findings from research on the brain and 
learning, the facilitators requested that attendees convene in role-alike groups. Each group was given 
a scenario to read that incorporated findings from brain research about what best supports learning. 
After reading their scenario, each group was to discuss current practices with regard to brain-based 
learning principles, what impediments stood in the way of implementing such practices, and how 
brain-based principles might be used to enable SMART learners. 

The groups read and talked until 1 1:35, at which point the facilitators requested that each 
provide a brief report of their conversation. Quest staff wrote participant comments under one of 
two columns (brain-based principles planned for implementation and impediments to 
implementation) on large pieces of butcher paper. 
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At 1 1 :46, the facilitators broadened the discussion, asking attendees, “What ah-ha or insight 
did you develop as you read and discussed,” and then writing their replies on a Mindmap, a visual 
depiction of how ideas are interconnected. Much of the discussion centered on how to incorporate 
the practice, supposed to enhance brain functioning, of drinking water throughout the day into the 
classroom without undue disruption. 

Participants were dismissed for lunch at 12:10. During lunch, many attendees noted the 
falling snow; several teams made plans to leave the rally early in order to avoid driving in inclement 
weather in the evening. Attuned to participant concerns about the worsening weather. Quest staff 
decided to adjourn the rally following lunch. 

Reconvening the group at 12:54, the facilitators announced that the rally would be adjourned 
shortly. Participants were asked to complete evaluation forms, indicate dates during which they 
would not be able to attend the next rally, and provide to Quest staff completed worksheets 
documenting which school improvement goals school teams intended to align with project activities. 
The facilitators then thanked attendees and wished them safe travels home. 
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FINDINGS 

Achievement of Rally Goals 



The extent to which rally goals were achieved at the February elementary school network 
rally is discussed in this section. Data are drawn from participant observation, two open-ended 
feedback forms, one form soliciting both qualitative and Likert-type quantitative feedback, and 
informal interviews. 

Participants were requested to complete three feedback forms at the rally. Twenty-three of 
the 28 attendees completed the first form, distributed at the end of the first day’s activities, 
representing a return rate of 82%. Two forms were distributed at the conclusion of the rally. One 
form posed only open-ended queries; 21 participants completed this, for a return rate of 75%. The 
second posed both qualitative and quantitative questions; 20 completed this questiormaire, 
representing a return rate of 71%. Response rates were likely impacted by the early exit of several 
participants after the first day of activities and by disorganization caused by the early dismissal of 
the rally. 

The open-ended final feedback form asked participants to denote their role. Of the 21 
respondents who completed this form, 12 were teachers, four parents, and five school administrators. 
Too, attendees were asked how many prior Quest events they had attended. Four were new 
participants and had attended no previous events, although they were employed by schools already 
in the network. Ten had participated in one or two other Quest gatherings, while seven had been 
involved in three or more. 

Participants were asked on one of the two final evaluation forms to rate the degree to which 
each of the six rally goals had been met, using a 5 -point Likert-type rating scale offering response 
options from 1 (“not accomplished”) to 5 (“extremely well accomplished”). The results are displayed 
in Table 1. Responses received ranged from a low of 2 to a high of 5. Cronbach’s Alpha revealed 
that the scale possessed sufficient internal consistency reliability at .88. 

Goal One, to cormect with colleagues, and Goal Five, to commit to continue learning with 
the Quest community, both received the highest mean ratings of 4.65 and the smallest standard 
deviations of .49 (see Table 1). These statistics indicate that participants thought with a fair amount 
of consistency that both goals had been well achieved. Least highly rated were Goal Six, to commit 
to continue the Quest back at school, with a mean of 4.40 (SD .75), and Goal Four, to create personal 
and shared meaning, with a mean of 4.45 (SD .60). Nonetheless, even these two means indicated 
that respondents felt the two goals had been well accomplished. Overall, there was only a difference 
of .25 between the highest and lowest mean ratings, another indication that participants felt all the 
goals had been well achieved at the rally. 
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Table 1 



Participant Rating of the Achievement of Rally Goals 



Goals 


N 


Mean 


SD* 


Goal One: Connect with colleagues 


20 


4.65 


.49 


Goal Two: Create a learning community 


20 


4.50 


.61 


Goal Three: Connect with concepts concerning school improvement 


20 


4.55 


.60 


Goal Four: Create personal and shared meaning 


20 


4.45 


.60 


Goal Five: Commit to continue learning with the Quest community 


20 


4.65 


.49 


Goal Six: Commit to continue the Quest back home 


20 


4.40 


.75 



* Standard Deviation 



Goal One: Connect with Colleagues 

This goal received one of the two highest ratings from participants on the quantitative 
assessment of goal achievement. With a mean of 4.65 and a standard deviation of .49, respondents 
thought with a fair level of consistency that the goal of connecting with colleagues had been met 
quite well at the rally. 

Asked on the final open-ended feedback form in what ways they had connected with 
colleagues at the rally, one respondent did not reply to the query, five offered multiple replies, and 
15 offered one response. Seven respondents mentioned the value of sharing stories and experiences, 
while five noted connecting through exchanging ideas. Working toward similar goals and feeling 
closer to one another were themes that enhanced connection with colleagues in three responses each. 
As one participant put it, “I feel so much more a part of the whole and less isolated.” Two replies 
indicated that connection with colleagues was facilitated by their sense of enthusiasm and 
“inspiration,” as one phrased it. One attendee simply wrote that connection was “very good.” Four 
responses were idiosyncratic and thus could not be coded. Of these, one respondent reported gaining 
“insight into several situations," and another felt connected to other colleagues through 
“understanding the roles that each of us plays in educating our children.” 

Participant observation seemed to corroborate feedback form data indicating that attendees 
made satisfactory connections with each other. Interactions appeared respectful yet relaxed, and 
participants seemed to share their personal and professional perspectives with each other easily. And 
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participants found unique ways to connect with each other. For example, one school team presented 
a video about their school that they had produced in conjunction with students. 



Goal Two: Create a Learning Community 

With a mean rating of 4.50 (SD .61), respondents expressed their sense that the goal of 
creating a learning community had been met well at the rally. 

On the open-ended feedback form, participants were asked whether Quest network members 
became more of a learning community at the rally and, if so, to provide some specific examples of 
how this happened. Three respondents did not answer this question. Sixteen replied that the 
learning community had been strengthened at the rally. Of these, 15 provided one or more examples 
of how the learning community was enhanced. Eight responses indicated that the content presented 
during activities had helped bolster the Quest learning community. The collaboration engaged in 
during various activities and discussions enhanced the learning community for three respondents, 
and the personal connections rendered over time were important for three other participants. As one 
such respondent put it, “In coming to know members better the community is strengthened.” One 
comment noted the value of sharing stories of success to building the learning community, and a 
final response suggested that the respondent’s own concern to “be more active in progressing our 
school forward” was important. 

Two replies to the question were more ambivalent about the building of the learning 
community. One respondent wrote, “Since only two of us were able to attend this rally, we did not 
feel that we became more of a learning community, at this rally[.] [Hjowever, we do plan to involve 
more faculty and parents.” This comment seems to suggest that the respondent interpreted the query 
in terms of the school team rather than the Quest network as a learning community. A second 
attendee wrote of being “not sure” whether the learning community had been enhanced, but added 
that “inclusion of out of school parents must make some difference.” 

During an informal interview, a teacher implied that the Quest learning community 
continued to grow, reporting that she found this to be the “easiest of all the rallies to talk in groups 
. . . to express my own feelings.” Her comment suggests that, for this participant, rallies were 
becoming increasingly comfortable venues for discussion and learning with colleagues. 

After the first day of the rally, attendees were requested to complete a feedback form. 
Respondents were asked on this fomi in what ways they had contributed to the day’s activities and 
interactions. It can be argued that contributions to the day’s events also augmented the growing 
Quest learning community. Of the 19 respondents who replied to this query, 13 wrote of engaging 
in discussion and offering their perspectives. Two of these added that their contributions were 
“sincere” or “honest.” Two replies mentioned the dynamic of both listening and discussing, while 
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three offered that their major contribution was assisting with the demonstration of the Protocol 
process. 

Participant observation revealed various indications that attendees experienced a sense of 
community during the rally. Appreciative laughter and applause, Fred’s willingness to share what 
he had learned through use of the Protocol process, attendees’ apparently honest critiques of the 
Quest self-scoring profile, and their attentiveness to the demonstrations of the Protocol by both 
teachers and students all appeared to suggest that participants experienced and contributed to the 
Quest learning community. 



Goal Three: Connect with Concepts Concerning Continuous School Improvement 

Receiving a mean rating of 4.55 (SD .60), the goal of connecting with concepts related to 
continuous school improvement seems to have been achieved well at the rally. 

In reply to a request on the final open-ended evaluation form for descriptions of specific 
content related to continuous school improvement with which they connected at the rally, 17 
respondents mentioned at least one such concept. Eleven participants v^rote of connecting with one 
of the techniques for engaging school community members that was demonstrated during the rally. 
Six replies indicated connection with information provided about brain-based learning principles; 
two replies with plans made to use the techniques learned; one reply with the SMART learner 
concept; and one reply with "the power of vision and positive leadership.” 

One participant, during an informal interview, mentioned specific content with which she 
connected during the rally. She reported that the "brain-based [information] was very exciting, very 
interesting . . . also the Protocol." 

Attendees were also asked on the mixed method final feedback form for their assessments 
of how specific activities and content "worked” for them. Of the 20 respondents to this form, 14 
offered positive evaluations of the Protocol process demonstration. "Vivid and real motivating,” 
wrote one such respondent. Two respondents gave unique replies, one noting that he had 
participated in the demonstration and the other that the Protocol process appeared “similar to our 
peer conferencing.” One comment could not be interpreted, and three respondents to the form did 
not reply to this query. There were no negative assessments of the demonstration of the Protocol 
process. 

The comments of the team demonstrating the process certainly seem to indicate their 
connection with it, and the benefits accruing to their school as a result of its use. Students 
themselves also appeared to connect with the process and with their own writing through the process. 
One of their teachers reported that students engaging in the Protocol process now appeared to think 
of their writing as “their own” rather than as perfunctory completion of assignments. 
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On the other hand, when asked during the rally by facilitators to discuss ways in which the 
Protocol process might be applicable in their own schools, only one team reported finding a potential 
use for it. This perhaps signifies that attendees did not connect quite as well with the process as 
feedback form data suggest. 

Assessments of information provided about brain-based learning research were likewise 
positive. Twelve respondents gave positive evaluations of this information. Comments included, 
“[I] appreciate the info [sic] and bibliography," “I’m really interested in learning more," and “Really 
made me think about how I run my classroom and will need to make improvements in some areas.” 
Four comments were difficult to interpret, although three seemed to mention small pieces of 
information associated with the review of brain-based learning research. One comment was 
somewhat critical of the review: “I wish there had been more concrete info [sic] on this rather than 
general discussion of ideal situations." And three respondents to the feedback form did not reply to 
this question. 

Asked on the mixed-method feedback form how the review of school improvement plans 
“worked” for them, two respondents offered critiques. One such participant felt there was “not 
enough time" for completion of the activity, while the second simply wrote that the review “didn’t 
work at all.” Two more replies were somewhat ambivalent, both noting that their school teams had 
already undertaken such review beforehand. Equivocally, one participant wrote, “I was aware.” Five 
respondents to the form did not reply to this prompt. On the other hand, the ten remaining 
respondents appraised the review of school improvement plans positively. Comments included, 
“The time spent reviewing and documenting our progress was helpful," “Good reminder of what we 
need to be doing,” and “[Made] us reevaluate where we are headed and come up with a more specific 
plan.” 



Eleven respondents to the final mixed-method feedback form did not reply to the query about 
how the use of the Quest self-scoring profile “worked,” creating the highest rate of nonresponse on 
the form. This may be due to the relatively short amount of time devoted to this activity, which in 
any case was intended to provide editorial feedback to Quest staff rather than to generate data for 
attendees. Four replies indicated positive assessment of the use of the profile, while four others were 
more ambivalent. Among those ambivalent comments were “I’m still unsure how to use the results,” 
and "I felt as if I had answered these questions before." The other two ambivalent comments 
suggested that the profile was too long and could be abbreviated. A final reply was difficult to 
interpret, although it appeared to address the purpose of the profile: “Take a look at where we are 
in our school improvement." 

Another indication of content with which participants connected includes responses to a 
query on the first feedback form distributed following the first day of the rally. Asked what “things 
I want to explore further,” 12 of the 19 respondents to this prompt mentioned wanting to learn more 
about using the Protocol process in their schools or classrooms. The remaining replies were 
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idiosyncratic and could therefore not be coded together. These unique responses included: “Dottie’s 
school,” “more team time for planning,” “use of data/information from surveys,” “how to implement 
the ideas at our school,” “communication processes,” “how to build a learning community. From the 
perspective of parents,” and “assessment.” 



Goal Four: Create Personal and Shared Meaning 

The achievement of the goal of creating personal and shared meaning received the second 
lowest mean rating. With a mean of 4.45 (SD .60), however, respondents felt that the goal had been 
well achieved. 

Asked on the final open-ended feedback form to name two or three personal or shared 
meaning that had been created at the rally, 12 respondents chose not to reply. The responses of the 
nine participants who did reply were multiple and diverse. Four mentioned a learning principle 
based on brain research. Four responses indicated the meaning generated through discussions with 
other participants, while one simply gained “insight into our school and our basic problems.” 
Another wrote of coming to believe that “the idea is also the possible,” and that “there are creative, 
intelligent, dedicated educators in every situation, and I can benefit greatly from interacting with 
them.” 



Participants were also asked on an evaluation form distributed at the end of the first day of 
the rally to describe any “learnings, insights, [or] ah-ha’s” they gained. Of the 20 respondents who 
replied to the query (two of whom offered multiple responses), 13 noted the value of learning about 
the Protocol process. Of these, one mentioned the meaning he gained from observing the process: 
“Protocol process in classroom can be used in micro reflection.” Three respondents wrote of having 
made more meaning around Quest itself “The whole Quest concept and how it related to our 
children’s learning” was meaningful to one such attendee, while another reported gaining a “better 
understanding of Quest.” Student participation during the demonstrations of the Protocol process 
were meaningful to three respondents, as were the opportunities for sharing amongst participants for 
three more respondents. One attendee cited the value of working as a school team on school goals, 
and one wrote of reflecting on the notion that “we are all different, but the same.” 



Goal Five: Commit to Continue Learning with the Quest Community 

Seven respondents did not reply to the question, “To what extent do you feel committed to 
continuing to learn with the Quest community? For what reasons do you feel this?” on the final 
open-ended feedback form. Of the 14 who did respond, three provided multiple answers. Three 
replies indicated some level of enjoyment of working with AEL toward continuous improvement. 
Two responses each suggested that commitment to the project was enhanced by the benefits and the 
information received during Quest interactions. The personal connection with project staff were 
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important to the sustenance of commitment for two respondents. The remaining replies were 
idiosyncratic. Sharing ideas, learning together, meeting others, and experiencing personal renewal 
were significant themes in these responses. Other themes included the enthusiasm of Quest staff, 
the leadership provided by AEL, and the ongoing nature of the project. “I feel committed because 
we have come this far and are not finished,” said one participant. 

In corroboration with the written feedback, participants gave the achievement of this goal 
one of the two highest mean ratings, 4.65 (SD .49), on the 5-point Likert-type scale. 

Goal Six: Commit to Continue the Quest Back Home 

The achievement of the goal of committing to continue the Quest for continuous school 
improvement back at their schools received the lowest mean rating from participants. Nonetheless, 
with a mean of 4.40 (SD .75), respondents deemed this goal had been well met. 

Asked on the final open-ended feedback form in what ways they were committed to 
continuing the Quest for school improvement back at their schools, 14 respondents offered brief 
descriptions of various plans for action, one of whom offered multiple responses, and seven did not 
reply. Five responses indicated that respondents intended to participate in planning activities, from 
“developing a comprehensive school improvement plan” to “work[ing] on school vision.” Sharing 
information with others and using techniques learned at the rally were ways in which three 
respondents each intended to express their commitment to continuous school improvement at their 
schools. Two respondents noted specific activities they hoped to undertake once they returned to 
their schools. One simply reported being “very” committed to continuous improvement, and another 
wrote of being committed to “have SMART learners.” 

Suggestions for Improvement 



Participants offered few suggestions for improving the rally. For instance, asked on an 
evaluation form following the first day of activities to describe “things that would have allowed me 
to contribute more,” 16 of the 23 respondents did not reply and three provided answers that were 
intended as jokes rather than serious criticisms. The four remaining respondents suggested the 
following: “open groups,” “if I knew more about the project,” “more small group sharing,” and 
“overcoming my own feelings of shyness." 

Likewise, when asked what “things to trash,” 18 of the 23 feedback form respondents did not 
reply, and one offered a joking response. Two participants critiqued the coffee and the bathtubs, and 
one the conference room temperature. A final respondent would have liked to “trash” the “extremely 
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lengthy discussions,” suggesting that, at least for this participant, more active sessions would have 
been preferable. 

Asked on the mixed-method final evaluation forms for their assessment of specific topics and 
activities, the overwhelming majority of respondents offered positive feedback. However, one 
respondent suggested that the presentation of material on brain-based learning research could have 
provided “more concrete” information. Four respondents to a prompt on the same evaluation form 
concerning the review of school improvement plans gave negative or ambivalent assessments about 
the activity as either redundant or given short shrift. And, more than half (1 1) of the 20 respondents 
to the questionnaire did not answer the prompt about the use of the Quest self-scoring profile. Of 
those who did reply, four offered ambivalent comments, two suggesting that the profile was too long, 
another that the questions were redundant, and a fourth that use of the profile results was unclear. 

Overall, however, participants did not make critiques of essential components of the Quest 
project or of the rally in interviews, during rally activities, or on evaluation questionnaires. 

Comparative Findings 



Quantitative data from earlier evaluation reports (Howley-Rowe, 1999a, 1998a) can be 
extracted to compare respondents’ ratings of the six rally goals across Quest events. Such 
comparisons will provide a means to evaluate the extent to which Quest events consistently meet 
project objectives. 

It is interesting to note that mean ratings of the achievement of rally goals have changed very 
little over time (see Table 2). Goals One through Four remained most stable across the three rallies 
assessed, with the largest mean difference appearing as .12 increase in the mean rating of Goal Four 
between November 1997 and November 1998. The two goals concerning commitment, on the other 
hand, showed larger mean differences over time. The mean for achievement of Goal Five, to commit 
to continue learning with the Quest community, decreased by .19 in November, 1998 and then 
increased by .14 in February 1999, nearly approximating the mean rating received in November 
1997. The most dramatic mean difference was for Goal Six, to commit to take the Quest for 
continuous improvement back to school, between November 1998 and February 1999, with a mean 
of .41. 

Overall, whJle mean ratings of five of the six goals declined between the November 1997 and 
February 1999 rallies, the differences are so minimal as to have no practical significance. And, the 
mean rating for Goal Four, creating personal and shared meaning, increased. In sum, it appears that 
the achievement of project goals across events has been quite consistent. 
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However, it should be noted that both high school and elementary network members 
participated in the November 1998 rally, and thus the data represent participants from both networks. 
Unfortunately, data were not disaggregated by school level. Another impediment in comparing 
mean ratings is that no quantitative data were collected at the February 1998 rally. 



Table 2 



Mean Ratings of Achievement of Goals Across Rallies 





November 1997 


November 1998 


February 1999 


Goals 


N 


Mean 


SD* 


N 


Mean 


SD* 


N 


Mean 


SD* 


Goal One 


33 


4.71 


.50 


37 


4.65 


.68 


20 


4.65 


.49 


Goal Two 


33 


4.57 


.55 


37 


4.51 


.56 


20 


4.50 


.61 


Goal Three 


33 


4.58 


.64 


37 


4.51 


.73 


20 


4.55 


.60 


Goal Four 


33 


4.39 


.75 


37 


4.51 


.61 


20 


4.45 


.60 


Goal Five 


33 


4.70 


.52 


37 


4.51 


.65 


20 


4.65 


.49 


Goal Six 


33 


4.76 


.49 


37 


4.81 


.46 


20 


4.40 


.75 



* Standard Deviation 
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CONCLUSIONS AND RECOMMENDATIONS 
Conclusions 



Several conclusions about the February 1999 elementary school network may be made based 
upon the data collected. 

It can be concluded that participants made satisfactory connections with their colleagues at 
the rally, as per Goal One. Furthermore, attendees continue to value and enjoy such relationships. 
Some participants also expressed appreciation of their personal connections with Quest staff 

The Quest learning community continued to be supported and nurtured by network members 
at this rally through the content shared, collaboration, and personal relationships. It can, therefore, 
be concluded that Goal Two, creating a learning community, was met at this rally. 

Participants connected with a variety of content at the rally, especially information on brain- 
based learning and the demonstrations of the Protocol process. Moreover, attendees found such 
content very useful. Goal Three, connecting with content related to continuous school improvement, 
thus seems to have been achieved. 

There was a relatively low response rate (57%) to an open-ended question concerning 
specific meaning participants created during the rally, perhaps indicating that respondents did not 
fully understand the query or were unwilling to communicate the meanings they created. On the 
other hand, a few participants did generate some personal meaning around continuous school 
improvement. It can be concluded Goal Four, to create personal and shared meaning, was met, 
although the low response rate confounds this conclusion somewhat. 

Overall, participants continue to be committed to the Quest learning community, as per Goal 
Five. This commitment was supported by a wide variety of Quest characteristics and structures, 
including personal connections with project staff, learning as a group, and sharing ideas and 
resources. 

According to quantitative data, attendees were least certain about their level of commitment 
to take the Quest for continuous school improvement back to their schools. Goal Six. Qualitative 
data, however, do not suggest that participants felt this goal was achieved to a significantly less 
extent than were other goals. It can be concluded, then, that participants were at least somewhat 
committed to continuing the journey of school improvement within their local contexts, although 
some of their plans to do so seemed ambiguous or provisional. In part, this may be due to the early 
dismissal of the rally, eliminating much of the time intended for school team planning. 
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Project goals were achieved with some consistency across the Quest elementary school 
rallies assessed. The extent to which the two goals concerning commitment— to continue learning 
with the Quest community and to take the Quest for continuous school improvement back to 
school— were achieved across elementary rallies fluctuated more than achievement of the other four 
goals. It could be concluded that elementary network members find commitment to Quest and to 
continuous school improvement somewhat more troublesome to sustain over time and under shifting 
circumstances. 
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Recommendations 



Analysis of the data leads to several recommendations for Quest staff to consider as they 
continue to design rallies. 

Commitment to continue the Quest for ongoing school improvement at network schools 
appears to be one of the more difficult goals to sustain over time and in local situations vulnerable 
to change. For upcoming rallies, Quest staff should consider offering structures or techniques for 
supporting schools as they undertake improvement projects. The Protocol process demonstrated at 
the rally and evaluated in this report appears to have been very valuable to participants interested in 
data collection and ongoing collegial discussion. However, project staff may also offer rally 
participants tools for maintaining commitment over time, such as enhanced use of the Quest listserv, 
processes for brokering change, and evaluation methods. 

Because participants value connecting with colleagues and Quest staff so highly, staff should 
continue to offer both structured and informal venues for such connection. Means of sharing ideas 
and stories could include continued use of various groupings of attendees at events, time for school 
team work, opportunities for introductory activities, participant contributions to Quest publications, 
and structured communication via the project listserv. 

Opportunities for network members to participate actively in the creation and support of the 
Quest learning communit)^ ought to be extended. Thus far, such opportunities have included 
presentations by school teams at project events, co-authoring presentations or publications with 
Quest staff, and the Scholars program. Other ways to offer occasions for network members to 
nurture the learning community might include support for members to attend co-ventures in learning 
to other network schools, invitations for participants to develop content for rally presentation, and 
asking veteran network members to conduct introductory sessions for new attendees. 

Another recommendation is that Quest staff continue to provide content about continuous 
school improvement to network members. As in the past, such content can range from specific data 
collection and communication techniques, to research-based suggestions for classroom practice, to 
theoretical perspectives on school change. Too, project staff may want to assess network members’ 
interests and concerns at various points to determine what content might be most useful to them. 

It is unlikely that participants who report a sense of renewal or inspiration as a result of 
participating in a Quest event do not also create meaning around the content discussed. However, 
many attendees do not provide written accoimts on evaluation forms of such meaning. Quest staff 
might therefore consider evaluating the achievement of the goal concerning meaning-making more 
formally via interviews rather than questionnaires. 
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In addition, Quest staff could offer further opportunities for participants to facilitate more 
explicitly the development and articulation of meaning around continuous school improvement. Such 
opportunities might involve debriefing sessions following project events or the publication of a 
Quest Log devoted to participant accounts of personal and shared meanings created at events. 

Given that participants at prior events reported that action planning within their school teams 
would be a valuable activity (Howley-Rowe, 1998d, 1998e), Quest staff may want to devote more 
time near the beginning of rallies for school teams to convene. Such a structure might have 
enhanced participants’ sense of commitment to continue their ongoing school improvement efforts 
despite the early dismissal of the rally. 
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Quest Brochure and Framework for Continuous Improvement 
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Quest for Quality Learning Communities 

A Program for Continuous School Improvement 



School improvement is challenging work; to be 
effective, it must be continuous. Improvement is 
not a single act or program; it is a process of al- 
ways wanting to learn more about how better to 
help all students achieve at higher levels. Improve- 
ment is visionary; it involves risk-taking, uncer- 
tainty, and a rejection of “doing what we Ve 
always done.” Most of all, improve- 
ment requires more than indi- 
vidual effort: it is a collabo- 
rative endeavor that en 
gages and responds to 
the diverse voices 
within an entire 
community. 

Teams from 20 
schools in a four- 
state region now 
collaborate with 
staff from the Ap- 
palachia Educa- 
tional Laboratory 
(AEL) to study and 
learn together in the 
Quest project, and each 
school takes a slightly different ^ ^ 

path. For example, one school tar- 
gets increased parent involvement; an- 
other hopes to raise the level of student thinking 
through teachers’ working together and coaching 
one another; a high school improves teaching by 
listening to what students say about how they 
learn best; other schools focus on specific curricu- 
lum areas such as writing or science education. 

The Quest framework unifies their thinking about 
school improvement. These core values offer a 
blueprint for continuous progress: ongoing ques- 
tioning of practice, high expectations for all, in- 
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dividual responsibility for better performance, col- 
legial sharing and support, and thoughtful reflec- 
tion on practice. 

Stemming from these values is a clearly defined 
vision of student excellence that is shared by all 
members of the school community. A strong 
learning culture encourages both stu- 
dents and teachers to choose con- 
tinuous improvement as a way 
of life in their school. Mem- 
bers of the school com- 
munity connect to one 
another through a 
shared commitment 
to improved learn- 
ing conditions for 
all. Shared leader- 
ship encourages and 
enables everyone to 
assume responsibility 
for making a positive 
impact on the school 
community. Shared goals 
for student learning moti- 
vate individuals to improve 
their performance and help focus 
the energies of the entire community. 

The collection, analysis, and use of student as- 
sessment data sustains continuous improvement, 
providing a measure of the effectiveness of the 
community’s efforts. SMART learners are Suc- 
cessful, Motivated, Autonomous, Responsible, 
and Thoughtful. Fully equipped to become life- 
long learners, they are ready for life and work in 
the 21st century. In short, continuous improve- 
ment spawns the energy and excitement neces- 
sary to transform a collection of individuals into 
a true learning community. 
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Goals of the Quest Project 



1 . Connect with colleagues. By serving on a Quest leadership team, participants connect with others on their school 
team, forming bonds that enhance working relationships. In addition. Quest teams connect with teams from 
other schools, districts, and states, allowing everyone to learn from others* experiences. A listserv, inquiry@ael.org, 
facilitates connections across the network. 

2. Create a learning community. Teams become part of the Quest net- 
work learning community with the expectation of recreating this ex- 
perience in their own community. 

3. Connect concepts and stories related to continuous school im- 
provement. At Quest rallies, the Quest framework is a source of study, 
dialogue, and sharing among teams. 

4. Create personal and shared meaning. The Quest network places a 
high value on processes such as reflection and dialogue, which lead to 
deeper understandings of continuous improvement. 

5. Commit to continue learning with this community. Quest schools 
have made a three-year commitment to study and learn together, with 
a focus on improving student achievement. 

6. Commit to continue the Quest back home. The “rubber hits the road” at schools, not at Quest events. AFT, helps 
school teams take their learnings home and apply them for the benefit of students. Site visits, called Co-Ventures 
in Learning, provide opportunities for AEL staff to visit each school, in order to better understand the context of 
that schoofs efforts, and tailor assistance to the school’s needs. 

The Quest project hopes to achieve results at three different levels: 

• For individuals, sharing leadership on a Quest team leads to more reflective practice and renewed under 
standing of the concepts that support continuous improvement. 

• For schools. Quest will provide motivation and support for ongoing and/or new school-based initiatives to 
improve teaching and learning. 

• For the Quest network of schools, our collaborative learning and research will yield stories, insights, 
processes, and products — all of which will be helpful to the broader educational community. 
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contact Sandra Orlctsky or 
Beth Sanes (sanesb@acl.org) at 



P. O. Box 1348, Charleston, WV, 25325 
800-624-9120 y 

\www.ael.org/rel/quest/index/htnv/ 



What Is a learning community? 
“Learning communities are essen- 
tially communities of Inquirers . . . 
sustained by a continued commit- 
ment to share this Journey of 
exploration with one another on 
matters people care deeply about” 
(Ryan, 1995). 



Peter Senge et al. (1 994) write that a 
learning organization “Is a place 
where people continually expand 
their capacity to create the results 
they truly desire, ... and where 
people are continually learning how 
to learn together.” 
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Inquiry Into Improvment 

Feedback Form 



The rally planners would appreciate your comments based upon the first day’s experience. 
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Quest Elementary School Network Rally 
February 22-23, 1999 

Please describe how the following “worked” for you? 



Demonstration of Protocol 


Review of school improvement plans 


Review of brain-based learning research 


Use of Quest Instrument for Continuous School 
Improvement: Self-Scoring Profile 



Now, using the scale below, please circle the number that best describes the degree to which each 
of the following six goals were accomplished during the rally. 





5=Extremely well accomplished 


3= Average 


l=Not accomplished 




Goal 1: 


To connect with colleagues 


5 


4 


3 


2 


1 


Goal 2: 


To create a learning community 


5 


4 


3 


2 


1 


Goal 3; 


To connect with concepts concerning 
continuous school improvement 


5 


4 


3 


2 


1 


Goal 4: 


To create personal and shared meaning 


5 


4 


3 


2 


1 


Goal 5: 


To commit to continue learning with 
this community 


5 


4 


3 


2 


1 


Goal 6: 


To commit to continue the Quest back 
home 


5 


4 


3 


2 


1 
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Quest Elementary School Network Rally 
February ll-l'i, 1999 



Quest staff continue to be interested in your experience at Quest rallies. Won 1 you please take a 
few minutes to complete the following questions? Your responses will remain anonymous and 
confidential, so please feel free to be as candid as possible. Thank you! 



1. In what capacity are you attending this rally? Please check one. 

Teacher Community member Other (please specify) 

Parent School administrator 

Student Support staff 

2. How many Quest events have you attended prior to this rally? 



3. In what ways do you feel that you connected with colleagues at this rally? 



4. Did Quest network members become more of a learning community during this rally? Give 
specific examples to support your answer. 



5. Describe examples of specific content related to continuous school improvement with which 
you “connected” at the rally. 



(over) 



6. If you were able to create personal and shared meanings at this rally, what were 2 or 3 of 
these meanings? 



7. To what extent do you feel committed to continuing to learn with the Quest community? For 
what reasons do you feel this? 



8. In what ways are you committed to continuing the Quest for continuous school improvement 
back at your school? 
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The Protocol: 

A Process for School Improvement Through Inquiry and Dialogue* 



What is Protocol? 

Protocol is an interactive process for engaging an entire school community in inquiry 
about the quality of the teaching and learning taking place at a school. The protocol is based 
on a set of assumptions and design principles mirroring features of powerful learning and good 
teaching that emerged from California Senate Bill 1274, “A Demonstration of School 
Restructuring in Public Education,” and from the work of Joseph McDonald. 

Assumptions 

The following assumptions serve as the foundation for the design of the Protocol process: 



>■ Inquiry, or asking questions, is an essential part of school improvement. 

>• People need help, i.e. structures and defined roles, to practice and improve their 
ability to engage in inquiry. 

>• Schools should have questions about students’ performance towards standards. 

>• The examination of student work must be part of a school’s inquiry. 

>• Teachers and all members of the school community need the opportunity to have 
honest conversations about their work and students’ work. 




► Feedback from multiple perspectives helps individuals and groups understand their 
work more fully. 

>■ When feedback is received and used as part of ongoing relationships, there is a greater 
probability that the feedback will lead to action. 

>• Reflection is an essential tool for learning. 

>■ When individuals are engaged in reflection about their own learning, they are more 
likely to take action on what they have learned. 

>■ The Protocol is most useful in an ongoing cycle of asking questions, examining 
evidence of student achievement, and taking action. 



This piece is reprinted from “Guidelines for the Protocol ‘97-’98,” shared by Joel Shawn, Spring 
1998, at a meeting of the School Change Collaborative in Chicago. 
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Benefits and Outcomes from the Protocol 
The protocol; 



>• provides a safe setting for relective dialogue about the work and progress of schools 
and districts. 

>• helps to keep change efforts focused on the specific learning needs of students. 

► provides an opportunity to demonstrate, through opebn, honest, and authentic 
analysis, how a school’s improvement efforts have impacted student achievmeent 
relative to agreed upon standards. 

>• provides an opportunity to receive feedback on school improvement efforts. 



>► 

> 



> 
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Roles for Protocol Process 



1. Facilitator 

keeps time, explains roles and purposes, and 
monitors. 

2. Analysts/Presenters 

present information clearly and concisely; 
share openly their analytical reflections; be 
explicit about the kinds of feedback desired; 
listen to the feedback; accept whatever they 
wish (discarding the rest); and make 
decisions about future work, based upon 
feedback and their own analysis. 

3. Reactors 

Listen attentively. Seek to understand. Ask 
clarifying questions when time is allotted. 
Attend to the questions at issue posed by 
analysts. Give feedback of both types: 
supportive statements and critical questions. 



Agenda for Protocol 



^Before the Protocol 

1 . Determine the question around which discussion and analysis will occur. 

2. Collect student work that will help you answer the question. 

If it s the first time for the group to do protocol, or if the reactors and the analysts don’t 
know one another, begin the process with an overview and introduction. Otherwise move 
directly to the analysis. 





Overview 

5 min. Facilitator welcomes the group and reviews the roles and the 

agenda. 

Introduction 

1 0-20 Presenters provide background information about the topic at hand. 

This is intended to set the context. For example, if the presenters 
are going to talk about student writing, they might present some 
background here about the state assessment instrument, how 
frequently students write, the kinds of prompts that are given, the 
length of student writing, the expectations for parent involvement, 
etc. 

5 min. Reflectors may ask questions for clarification. 


*Analysis 
15-30 min 


Presenters discuss with one another the problem; they give their analysis of the 
question or questions which were formulated before the protocol began, and for 
which they have collected student work to serve as evidence. 


5 min 


Reflectors (scholars) ask brief questions for clarification. 


*Feedback 
10-15 min 


Reflectors form small groups of 3-5 to provide feedback on the essential questions 
identified above. Feedback is given in two forms — ^warm and cool. The warm 
feedback is given first; these are supportive statements, accentuating the positive 
that was heard in the analysis. Next reflectors pose the cool feedback in the form 
of questions. 

In a large group, the reflectors can work at table groups, posting their feedback as 
the analysts wander around the room listening in. In a smaller group, the 
reflectors can talk to one another as the analsysts listen to their discussion. 



^Reflection and Next Steps 

10-15 min Presenters engage in reflection on the feedback they heard. They plan their next 
steps/revisions/new ideas that were triggered by their own thinking and by the 
feedback they heard.their thinking, etc. 

Reflection/Journal Writing 

5 min Presenters and reflectors record their impressions and learnings 

from the process. 

*Debrief 

5-15 min The presenters and reflectors engage in an open conversation about the process, 
what they learned, what they liked or disliked about the experience, etc. 
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Feedback 



Feedback should address the team’s essential 
questions. 

Feedback is given in two forms: 

Supportive Statements 
Critical questions 
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Citation Form 



I 

I 

I 

I 

I 

r 



fe Program Evatuation Standards (1994. Sage) guided the development of this (check one): 

request for evaluation plan/design/proposal 

evaluation plan/design/proposal 

evaluation contract 

Y evaluation report 

other: 



interpret the information provided on this form, the reader needs to refer to the full text of the standards as they appear in Joint 
mmittee on Standards for Educational Evaluation, The Program Evatuation Standards (1994), Thousand Oaks, CA, Sage. 



le Standards v/ere consulted and used as indicated in the table below (check as appropriate): 



:riptor 

Stakehoider Identification 


The Standard was 
deemed applicable 
and to the extent 
feasible was taken 
into account. 

X 


The standard was 
deemed applicable 
but could not be 
taken into account. 


The Standard was 
not deemed appli- 
cable. 


Exception was taken 
to the Standard. 


Evaluator Credibility 


X 








Information Scope and Selection 


X 








Values Identification 


X 








Report Clarity 


X 








Report Timeliness and Dissemination 


X 








Evaluation Impact 


X 








Practical Procedures 


X 








Political Viability 






X 




Cost Effectiveness 


X 








Service Orientation 


X 








Formal Agreements 


X 








Rights of Human Subjects 


X 








Human Interactions 


X 








Complete and Fair Assessment 


X 








Disclosure of Findings 


X 








Conflict of Interest 


X 








Fiscal Responsibility 


X 








Program Documentation 


X 








Context Analysis 


X 








Described Purposes and Procedures 


X 








Defensible Information Sources 


X 








Valid Information 


X 








Reliable Information 


X 








Systematic Information 


X 








Analysis of Quantitative Information 


X 








Analysis of Qualitative Information 


X 








Justified Conclusions 


X 








Impartial Reporting 


X 








Metaevaluation 


X 
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